# Unsupervised Preference Optimization

Mistral-ORPO-β is a 7B-parameter language model fine-tuned using the ORPO method based on Mistral-7B, capable of directly learning preferences without a supervised fine-tuning warm-up phase.